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5  Reason  Not  to  Worry  About  This  Problem 


5.  There  is  too  much  data  already  - 
soldiers  can’t  handle  it. 

4.  Bandwidth  is  not  a  problem 
(It’s  just  a  modulation  problem). 

3.  High  speed  radios  will  soon  permeate  the  battle  field. 

2.  Nothing  will  ever  replace  USMTF. 

1.  Industry  will  fix  this  problem. 
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More  Why  Not  Worry  About  This  .... 


From:  Computer  Networks  (3rd  ed.,  1996),  by  Andrew  S.  Tanenbaum: 

“In  the  race  between  computing  and  communication,  communication 
won.  The  full  implications  of  essentially  infinite  bandwidth  (although 
not  at  zero  cost)  have  not  yet  sunk  into  a  generation  of  scientists  and 
engineers  taught  to  think  in  terms  of  low  Nyquist  and  Shannon  limits 
imposed  by  copper  wire.  The  new  conventional  wisdom  should  be  that 
all  computers  are  hopelessly  slow,  and  networks  should  try  to  avoid 
computation  at  all  costs,  no  matter  how  much  bandwidth  that  wastes. 
In  this  section,  we  shall  study  fiber  optics  ...” 


This  statement  may  be  true  for  fiber  optics,  but  it  is  often  repeated  as 
a  general  rule.  However,  wireless  communications  is  bounded  by  the 
above  limits  and,  at  the  lowest  echelons,  is  often  a  limiting  factor  in 
effective  battle  command. 

In  other  words,  we  always  want  to  send  more  than  we  can. 
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Commercial  Information  Management  Is  Based  Upon 
Guided  (e.g.,  Wire  &  Fiber)  Communications 


Relative 

Performance 

Computer  Technology 
(Single  CPU) 

Communications 

Technology 

1970’s: 

100  nsec  /  instr.  (CDC  6600) 

56  Kbps  (Arpanet) 

1990’s: 

1  nsec  /instr  (Cray) 

1  Gbps  (Fiber) 

2000’s: 

. 1  nsec/ instr  (Multi) 

50  Gbps  (Fiber) 

Improvement: 

10  fold  /  decade 

50-100 fold/  decade 

...  and  50,000  Gbps  (50  Tbps)  is  attainable  with  current  fiber  technology. 
Bit  Error  Rates: 

99.64%  of  bit  errors  over  fiber  (telephone  system)  are  single  bit  errors1 
and  can  be  handled  by  a  simple  checksum.  With  an  ATM,  8-bit,  header 
checksum,  the  probability  of  not  detecting  a  bad  cell  header  is  10-20  or 
one  cell  every  90,000  years  at  OC-3  rates  (155.52  Mbps). 

(Note:  w/ 1  billion  ATM  telephones,  this  is  1000  cell  errors  /  year). 

Note:  This  is  NOT  a  Tactical  Internet  Environment. 

1.  Observations  of  Error  Characteristics  of  Fiber  Optic  Transmission  Systems;  CCITT  SG  XVIII,  San  Diego,  Jan  89. 
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In  the  Mobile  Wireless  Environments 
Computational  Power  Is  Outpacing  Communications  Power 


EXAMINE  THE  RATIO  OF: 


COMPUTING  POWER  (MFLOPS) 

COMMUNICATIONS  POWER  (Mbps) 


WIRED  (e.g.,  LAN)  vs.  WIRELESS  (e.g.,  radio)  ENVIRONMENTS 

COMPUTER  (MFLOPS)* 

COMM  SYS.  (Mbps) 

RATIO 

(MFLOPS/Mbps) 

Pentium  4/2000  (655) 

Pentium  111/550  (197) 

Pentium  233  MMX  (33) 

Gbit  Ethernet  (1000) 
Ethernet  (100) 

Ethernet  (10) 

.65  § 

1.97  a 

3.3  u 

Pentium  4/2000  (655) 

JJ 

JJ 

JTRS  (20) 

NTDR  (.375) 

SINCGARS  (.0064) 

n 

32.7  £ 

1  >746  3 

5,156  ^ 

The  Vector  Whetstone-97  Benchmark.  http://www.dl.ac.uk/TCSC/disco/Benchmarks/whetstone.html 


CONCLUSION:  FOR  BATTLE  COMMAND  AT  THE  FIGHTING  ECHELONS,  WHERE 
WIRELESS  COMMUNICATIONS  IS  THE  NORM,  WARRIORS  NEED  TO  COMPLEMENT 
COMMUNICATIONS  POWER  WITH  CAREFUL  INFORMATION  MANAGEMENT. 
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Observations  and  Basic  Points 


•  Just  as  important  as  the  actual  bandwidth  values 
are  the  huge  variation  of  the  bandwidth. 

•  These  parameters  have  significant  implications  for  building 
battle  command  systems  -  especially  at  the  lowest  echelons. 

•  In  the  battle  command  business,  one  must  consider  the  whole 
environment  when  addressing  the  problems  associated  with 
data  replication. 

•  0  is  a  valid  value  for  throughput  [aka  delay  =  oo]. 

•  Interesting  requirement: 

One  must  be  able  to  operate  when  throughput  =  0, 

Otherwise,  just  throw  away  your  battle  command  system. 

•  Usual  response:  Are  you  nuts? 
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Some  Assumptions  and  Tenets 


•  Computers  will  get  faster,  smaller  and  use  less  power. 

•  Tactical  communications  will  continue  to  be 

7i  Highly  variable  (Guided  &  Wireless;  0  bps  to  Gbps  ),  and 
71  Often  less  than  demanded 

•  There  will  be  more  information  to  send  than  can  be  sent; 
we  have  to  quickly  select  what  and  when  we  exchange. 

•  Warriors  under  fire  don’t  have  time  to  fuss  with  computers. 

•  We  can’t  afford  to  propagate  message-based,  legacy  systems 
(like  TF-XXI).  We  need  Common  Distributed  Computing 
Environments  with  Model-Based  Battle  Command. 

Therefore:  Information  Management  &  Distribution  Must  Be: 

7i  Automatic  (  Hands-Off,  Context-Based  ) 

7i  Adaptive  (  Context-Based,  Respond  to  Bandwidth  Conditions  ) 
7i  Affordable  (  Treated  as  a  Distributed  Computing  Environment ) 
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Affordability:  Ultimate  Goal  - 
Common  Environments  At  All  Echelons 
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MAJOR  PITFALL  OF  EVERY  ATTEMPT  TO 
AUTOMATE  BATTLE  COMMAND  TO  DATE 


THE  AUTOMATION  OF  MANUAL  TECHNIQUES 
Need  to  rid  ourselves  of  the  “E-Mail”  mentality 
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Model  vs.  Message-Based  Battle  Command  (BC): 

This  Is  What  Makes  Automation  and  Adaptation  Affordable 


MESSAGE-BASED  BC 

INFORMATION  FLOW  BASED  ON  AUTOMATING 
MANUAL  TECHNIQUES 
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AND  IDEAS 
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MODEL-BASED  BC 

INFORMATION  FLOW  BASED  ON  CHANGES 
TO  A  FORMAL  MODEL 
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Model-Based  Battle  Command:  Each  Node 
Maintains  A  Model  Of  Its  Perception  Of  The  Battlefield 


CONCEPTS 
AND  IDEAS 


I 


I 


APPLICATION 

PROGRAMS 


CONVERT  TO 

-  -  DATA  - 

ABSTRACTIONS 


APPLICATION 

PROGRAMS 


I 


Human  - 
Oriented 
Form 


Computer  - 

Oriented 

Form 


*  Data  Abstractions  Are  The  Medium  Of  Communications  * 

*  Exchange  Controlled  by  Active  Database  Triggers  (State  of  Database)  * 

*  Reasonably  Different  Perceptions  Of  The  Battlefield  Are  Allowed  * 

*  Synchronization  Is  The  Realistic  Control  Of  Differing  Perceptions  * 
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Some  Ideas  Up  To  This  Point 


•  Synchronization  Is  The  Realistic  Control  Of  Differing  Perceptions. 

•  What  Do  We  Want?  . . .  Perfect  Synchronization,  Of  Course! 

•  Are  We  Going  To  Get  It?  ...  Not  Likely  Most  Of  The  Time. 

•  Ok,  Look  At  Extremes:  +  Infinite  Bandwidth  -  Perfect  Synchronization. 

-  No  Bandwidth  -  Unknown  Synchronization. 

•  In  The  Commercial  World,  We  Tend  To  Build  Systems  Toward 

Perfect  Synchronization,  But  This  Is  Not  Realistic  For 
Most  Military  Environments. 

•  In  The  Military  Environment  We  Must  Be  Able  To  Handle  Both  Extremes, 
Infinite  Bandwidth  And  No  Bandwidth,  Plus  All  The  Interesting  Cases  In 
Between;  This  Is  A  Form  Of  Adaptive  Battle  Command. 

•  Therefore,  We  Must: 

-  Mix  Predictive  Modeling  with  Known  Synchronized  Information 

and  Keep  the  User  Aware  of  Which  Is  Which; 

-  Monitor  (Or  Measure)  Bandwidth  and  Control  Synchronization 

Based  on  Current  Communication  Resources. 
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Adaptive  Synchronization  Based  On  Bandwidth 


0  is  a  valid  value  for  throughput,  or  oo  for  delay! 


Concurrent  Databases  Controlled  Asynchronization  Unsynchronized  Databases 
No  Prediction  Required  Some  Prediction  Required  Updates  Via  Prediction 
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%  Predicted  Data  Growing 


Percent  Estimation 


Bandwidth  -  Prediction  Trade-off 


X 1 
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% 

100 


0 


Stand-Alone  System 
(e.g.,  Constructive 
Model  or  Wargame) 


Visualizing  where  one  lies 
on  this  graph  is  imperative. 


Nominal,  “Interesting”  Zone 


Enough  Bandwidth 
To  Do  All  Desired 
(e.g.,  Concurrent 
Databases) 


Bandwidth 
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What  Can  We  Do  To  Accomplish  This  Goal? 

Quit  Fighting? 


WHEN  TO 
TO  EXCHANGE 
INFORMATION 


PREDICTION 


HOW  TO 
EXCHANGE 
INFORMATION 


INFORMATION 

EXCHANGE 

(REPLICATION) 


GOAL:  BALANCE,  OR  “TUNE” 

INFORMATION  MANAGEMENT  WITH  NETWORK  RESOURCES 
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Information  Distribution  In 
Tenuous  Communications  Environments 


•  Objective:  intelligent  &  prudent  dissemination  of  information. 

( to  use  resources  wisely  - 

avoid  sending  any  wasteful  l’s  and  0’s). 

•  Key  Enabler:  Model-Based,  vs.  Message-Based,  Battle  Command. 

•  Requirements  -  Perform  Information  Dissemination  that  is: 

Automatic:  Hands  Off,  Context  Based. 

Adaptive:  Depends  on  Resources  (e.g.,  Bandwidth  or  Power). 

•  Technologies  and  Capabilities  to  Accomplish  this  Goal. 

•  Active  Database  Triggers  (now  part  of  most  software  agents). 

•  Promiscuous  Replication  (initiated  by  triggers). 

•  Innovative  Transport  Protocols  (e.g.,  performance  feedback). 

•  This  means  that  communications  information,  like  performance 
and  connectivity  data,  should  be  part  of  the  data  schema. 

UNCLASSIFIED  -  APPROVED  FOR  PUBLIC  RELEASE 
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Example  Of  Bandwidth  Driven 
Information  Synchronization 


SIMPLE  CASE:  POSITIONAL  AWARENESS 


A  GROUND  SPEED  OF  10  METERS  /  SECONDS  (22.5  MPH)  POSITION  RESOLUTION  VARIES 

WITH  AN  AVERAGE  NETWORK  DELAY  OF  10  SECONDS  WITH  BANDWIDTH  CONDITIONS 

MEANS  AN  UPDATE  EVERY  100  METERS  AT  BEST;  (Vehicle  Speed  and  Network  Delay) 

OTHERWISE,  ONE  WILL  JUST  BACKUP  THE  OUTPUT  QUEUE. 
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Active  Database  Concept 


Application 

Programs 


ALARMS 


Triggers 
vs.  Queries 


In  Some  Applications,  Update  Rates  Are  too 
Frequent  to  Reasonably  Identify  Situations  Via 
Manual  Queries.  Active  Databases  Incorporate 
a  Monitor  That  Allows  Predefined  Criteria 
(Triggers)  to  Be  Entered.  Incoming  Data  Are 
Checked  Against  the  Triggers  and  Set  Off 
Alarms  When  Criteria  Is  Met. 
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Active  Database  Triggers  Can  Invoke  Any  Action 


APPLICATION 

PROGRAMS 


APPLICATION 

PROGRAMS 


ACTIVE 

INFO-BASE 


ACTIVE 

INFO-BASE 


<=> 


ALARMS  REPLICATION  MECHANISM 

Incoming  Information  Causes  Triggers  Incoming  Information  Causes  Triggers 
to  Notify  Application  Programs.  to  Update  Remote  Database. 
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Automatic 


Functional  Block  Diagram 
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Application 
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Application 
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Adaptive 


Transaction 
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IP-Based 
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Some  Replication  Approaches  & 
Promiscuous  Replication 


Wi 


CURRENT  REPLICATION  TECHNIQUES: 

TIGHT  CONSISTENCY:  ALL  DATABASES  MUST  RECEIVE  THE  UPDATE 

BEFORE  ANY  DATABASE  CAN  APPLY  THE  UPDATE. 
RESULT:  ALL  DATABASES  PORTRAY  IDENTICAL  STATES 
AND  HAVE  “IDENTICAL”  AUDIT  TRAILS. 

LOOSE  CONSISTENCY:  LOCAL  DATABASE  CAN  APPLY  THE  UPDATE  BEFORE 

OTHER  DATABASES  RECEIVE  THE  UPDATE; 

UPDATES  ARE  QUEUED  AND  EVENTUALLY  PASSED  IN  ORDER; 
RESULT:  ASYNCHRONOUS  AUDIT  TRAILS. 


PROMISCUOUS  (CASUAL)  REPLICATION: 

LOCAL  DATABASE  CAN  APPLY  THE  UPDATE  IMMEDIATELY. 

REPLICATION  IS  A  FUNCTION  OF  THE  DATABASE  STATE  AND  META-INFORMATION. 

(e.g.,  TACTICAL  SIGNIFICANCE  AND  CURRENT  CHANNEL  PERFORMANCE.) 
COMMON  AUDIT  TRAILS  ARE  NOT  REQUIRED.  SYSTEM  PROVIDES  BEST  EFFORT. 

mamm a  amuuuj  m  pbbcip  warn 
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Innovative  Protocol  Mechanisms 


•  Stack  Cognizance  -  Sharing  information  between  ISO  layers 

71  E.g.,  Signal  exchange  between  Transport  &  Datalink  layers. 

7i  Performance  feedback  to  applications  (throughput,  delay). 

71  MTU  size  to  applications  to  allow  context  blocking  within  MTU. 

•  “Just-in-time”  Packet  Construction 

71  Context  Blocking  -  packing  several  small  Application  PDUs  into  an  MTU. 

7i  Staleness:  property  of  data  -  duration  of  usefulness. 

7i  Assumes  that  access  delay  may  be  greater  than  staleness  value. 

71  Transport  Layer  builds  and  passes  segments  when  signaled 
by  lower  layer  (e.g.,  datalink  layer  for  CSMA). 

•  Overhearing 

71  Network  ->  Transport  Layer  pass-through  of  packets  not  addresses  to  host. 
71  Overheard  packet  marked  accordingly. 

•  Potential  application  to  Stream  Control  Transport  Protocol  (RFC  2690). 
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Example:  “Just-in-time”  Packet  Construction 


FEX:  Fact  Exchange  -  A  self-contained,  meaningful,  atomic  Application  PDU 

PRIORITY  QUEUE 


Status  Data 
to  Database 


Data-Ready  Signal’ 


‘  Channel-Ready  Signal 


* 


New  Data  With 
Priority  and 
Staleness  Value 

Assign  ID, 
to  report 
Success/Failure 


*  =  Waiting  for  ACK 
RTT  not  yet  expired 


New  Outgoing  Packet 


INTERNET  LAYERS 
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Resilient  Data  Transport  Features  - 
Interface  with  Network  Management 


•  Expects  that  there  are  more  data  to  send  than  can  be  sent; 
Goals: 

71  Never  Send  An  Unnecessary  1  or  0. 

71  Always  Send  Most  Important  Information  First. 

71  Expect  Delays  and  Failures. 

71  Report  Delays  and  Failures. 

•  Interface  Data  Unit  Includes 

Priority  and  Stateness  Values  for  Local  Use. 

•  Datagram-Oriented. 

71  Multi-destination  (reliable)  and 
Broadcast  (unreliable)  Supported. 

7i  Concatenation  Expected  by  Lower  Layers  (e.g.,  188-220A). 
71  Overhearing. 

•  Just-in-Time  Packet  Construction  (Postpone  Selection). 

•  Performance  Measurements  Maintained  and  Reported. 
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Adaptation  -  Collection  Of  Channel 
Performance  Data  (Delay  Metrics) 


Feedback  Loop: 
Data  Available 
For  Triggers. 


Delay  Values 
Fed  Back 
To  Database 
With  Channel, 
Host  &  Priority 


Status  Data 
to  Database 
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Pri  0 


PRIORITY  QUEUE 


Pri  1 
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ACK  623 


ACK  633 


ACK  924 


ACK  475 


Pri  2 


FEX  242 


FEX  175* 


FEX  133* 


FEX  129* 


Pri  3 


FEX  251 


FEX  142* 


FEX  138* 


FEX  123* 


PACKET  BUILDING  ROUTINE 


Channel 
Access  Delay 


Data-Ready 


Channel-Ready  Signal 


INTERNET  LAYERS 
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New  Data  With 
Priority  and 
Staleness  Value 

Assign  ID, 

Success/Failure 

Report 


Queue  Delay 
By  Priority 


Host 
RTT  Delay 


*  = 


=  Waiting  for  RTT 


New  Outgoing  Packet 
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Collection  of  Channel  Performance  Data 
( Various  Delay  Metrics  ) 


•  Channel  delay  data  collected  and  reported  back  to  database 
for  use  in  trigger  mechanism  (just  another  parameter): 

71  Average  Channel  Access  Delay  for  Each  Channel  Transmission. 

71  Average  Round  Trip  Time  for  Each  Host. 

71  Average  Queue  Delay  for  Each  Exchange. 

•  Use  delay  values  and  failure  reports  (reasons)  to: 

71  Compute  Optimal  Reporting  Frequencies  (see  example). 

7i  Tune  Information  Exchanges  to  Balance  Channel  Loading. 

71  Check  for  Stable  Queue  Sizes  and  Adjust  Dynamically. 

•  Passively  attempt  to  identify  and  predict  communication 
connection  and  congestion  problems. 

71  Look  For  Patterns  of  Concern  -  by  Channel  and  by  Host. 

7i  Include  A-priori  Information  (e.g.,  Battle  Plan) 

UNCLASSIFIED  -  APPROVED  FOR  PUBLIC  RELEASE 


Realistic  Handling  Of  Failed  Replications 


Concept:  Track  &  Handle  Replication  Successes  Separately  from  Replication  Failures. 
Successes  allow  one  to  know  what  others  know  about  you. 

Failures  used  to  derive  state  of  communications  system  and  potential  problems. 
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Communication-Based  Transaction  Failures 


*  Staleness  Timer  Expires  && 

Congestion 

Number  of  Transmissions  =  0 

*  Staleness  Timer  Expires  && 

Congestion  or 

0  <  Number  of  Transmissions  <  Max 

Channel/Host  Failure 

*  Number  of  Transmissions  >  Max 

Channel/Host  Failure 
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Summary:  Relationship  Between  the  Three 
Processes  Using  Model-Base  Battle  Command 
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MANAGES 
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2.  REPLICATION 
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SUCCESSS/FAIL  AND 
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STATISTICS  (e.g.,  DELAY) 
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( DELAY ) 
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BE  USED  FOR  EVENT 
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TO  INTERNET 


TI 

Tactical 

Internet 
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Summary  of  Resilient  Information 
Management  &  Distribution  Features 


1.  Information  Synchronization  Can  Automatically  Adjust  to  Bandwidth  Variations  by: 

A.  Including  Communications  in  the  Data  Model;  and 

B.  Passively  Collecting  Network  Performance  Statistics. 

2.  The  Active  Database  Triggers  Can  Now  Refer  to  the  Statistics  to  Dynamically  Vary 
Database  Synchronization  (  e.g.,  Position  Report  Resolution  Vary  With  Network  Delay  ). 

3.  Audit  Trail  Requirements  Are  Relaxed  Between  Databases  to  Ensure  that  the  Most 
Important  Information  Gets  Distributed  (  First )  -  l.e.,  Promiscuous  Data  Replication. 

4.  Because  a  Battlefield  Model  Exists  at  Each  Node,  Sophisticated  Application  Programs 
Can  Be  Used  to  “Fill  Holes”  Via  Prediction. 


Concurrent  Databases  Controlled  Asynchronization  Unsynchronized  Databases 


“BIG  COMM  PIPE”  “SMALL  COMM  PIPE”  “NO  COMM  PIPE” 
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%  Predicted  Data  Growing 


For  More  Information 


Dr.  Sam  Chamberlain 

Computational  &  Information  Sciences  Directorate 
U.S.  Army  Research  Laboratory  (ARL) 

Director,  USARL 

ATTN:  AMSRL-CI-CT (Chamberlain) 

APG ,  MD  21005-5067 

Phone:  410-278-8948  (DSN  298);  Fax:  2934; 

Email:  wildman@arl.army.mil 

URL :  h ttp://www.  arl.  army.mil/~wildman 


(  Papers  Online  ) 
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